<html>
<head><meta charset="utf-8"><title>rustbuild ci summaries · t-infra · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/index.html">t-infra</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html">rustbuild ci summaries</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="210270260"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270260" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270260">(Sep 16 2020 at 14:33)</a>:</h4>
<p><span class="user-mention" data-user-id="232545">@Joshua Nelson</span> btw, what I would <em>love</em> before we start changing this too much more is to work on getting "summaries" of what is being built in CI</p>



<a name="210270297"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270297" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270297">(Sep 16 2020 at 14:33)</a>:</h4>
<p>right now it's really hard for us to be sure we didn't randomly stop building something on some ci builder</p>



<a name="210270441"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270441" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270441">(Sep 16 2020 at 14:34)</a>:</h4>
<p>my thinking is we can upload a file containing a dump of the steps executed in rustbuild, like we already do with CPU utilization</p>



<a name="210270482"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270482" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270482">(Sep 16 2020 at 14:34)</a>:</h4>
<p>ah hmm and just make sure the <code>--dry-run</code> output isn't changing</p>



<a name="210270513"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270513" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270513">(Sep 16 2020 at 14:34)</a>:</h4>
<p>well we'd have the actual run results</p>



<a name="210270562"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270562" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270562">(Sep 16 2020 at 14:35)</a>:</h4>
<p>for now I wouldn't try to check that they're the same, that seems hard/annoying, but I would like to be able to easily diff two builds</p>



<a name="210270612"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270612" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270612">(Sep 16 2020 at 14:35)</a>:</h4>
<p>if we did <code>--dry-run</code> we could make this a unit test ;) and run it at the very beginning without having to do a full build</p>



<a name="210270642"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270642" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270642">(Sep 16 2020 at 14:35)</a>:</h4>
<p>well rustbuild already executes a full dry run right before</p>



<a name="210270749"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270749" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270749">(Sep 16 2020 at 14:36)</a>:</h4>
<p>I forget if we had to disable it but at one point we were verifying that the dry run and real results of the build matched</p>



<a name="210270786"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270786" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270786">(Sep 16 2020 at 14:36)</a>:</h4>
<p>but unit tests seem hard? Not sure I follow what you mean there</p>



<a name="210270821"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270821" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270821">(Sep 16 2020 at 14:36)</a>:</h4>
<p>the way there are <code>dist</code> tests currently</p>



<a name="210270875"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270875" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270875">(Sep 16 2020 at 14:36)</a>:</h4>
<p>well, those aren't really "what is done on CI" or at least I don't think they match faithfully</p>



<a name="210270883"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270883" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270883">(Sep 16 2020 at 14:36)</a>:</h4>
<p>I guess it would be painful to do that for each builder</p>



<a name="210270904"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270904" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270904">(Sep 16 2020 at 14:37)</a>:</h4>
<p>it's hard to be sure we matched, too</p>



<a name="210270944"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270944" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270944">(Sep 16 2020 at 14:37)</a>:</h4>
<p>that's why I want to generate summaries and upload to s3 so we can analyze offline</p>



<a name="210270958"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270958" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270958">(Sep 16 2020 at 14:37)</a>:</h4>
<p>makes sense</p>



<a name="210270983"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210270983" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210270983">(Sep 16 2020 at 14:37)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/242791-t-infra/topic/rustbuild.20ci.20summaries/near/210270441">said</a>:</p>
<blockquote>
<p>my thinking is we can upload a file containing a dump of the steps executed in rustbuild, like we already do with CPU utilization</p>
</blockquote>
<p>how would I get the steps? Just store the output of <code>--dry-run</code> somewhere?</p>



<a name="210271104"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210271104" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210271104">(Sep 16 2020 at 14:38)</a>:</h4>
<p>if y'all fancy log parsing I'm pretty sure you can get a list of all the artifacts produced by each builder</p>



<a name="210271134"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210271134" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210271134">(Sep 16 2020 at 14:38)</a>:</h4>
<p>the "upload artifacts to s3" step should include a line for each generated file</p>



<a name="210271141"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210271141" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210271141">(Sep 16 2020 at 14:38)</a>:</h4>
<p>/me likes writing shell scripts</p>



<a name="210271326"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210271326" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210271326">(Sep 16 2020 at 14:40)</a>:</h4>
<p>hey, if you wanted to write a tool that I could just <code>./foo download-logs &lt;hash&gt;</code> that'd be great too</p>



<a name="210271392"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210271392" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210271392">(Sep 16 2020 at 14:40)</a>:</h4>
<p>right now I think the best way is ~180 clicks in github UI</p>



<a name="210271505"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210271505" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210271505">(Sep 16 2020 at 14:40)</a>:</h4>
<p>I wonder if they have an API for that ...</p>



<a name="210271534"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210271534" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210271534">(Sep 16 2020 at 14:41)</a>:</h4>
<p>they have, it requires a token though</p>



<a name="210271583"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/rustbuild%20ci%20summaries/near/210271583" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/rustbuild.20ci.20summaries.html#210271583">(Sep 16 2020 at 14:41)</a>:</h4>
<p>anyway, this doesn't need to block the contributor side of things I think, only switching CI to default profiles</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>